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PREFACE 


This  report  contains  the  results  of  a  meeting  held  at  the  Ogden 
ALC  to  evaluate  the  potential  for  transfer  to  the  other  ALCs  of  three 
software  packages  developed  by  the  Readiness  Control  Center  at  Ogden.  This 
meeting  was  hosted  by  AFLC  LO/XR,  and  took  place  January  13-15 >  1981. 

This  effort  was  directed  toward  providing  some  interim  management 
system  enhancements  to  the  System  Managers  while  the  long-range  Logistics 
Management  System  Planning  activity  is  dealing  in  depth  with  the  Weapon 
System  Management  area. 
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INTRODUCTION 


The  Air  Force  Logistics  Command  basically  manages  by  item  rather  than 
by  system.  There  is  a  recurring  need  to  aggregate  the  Command's  information  by 
weapon  systems  for  reporting  to  Congress  and  to  assess  the  impacts  of  funding 
and  deployment  changes.  Few  management  systems  currently  exist  to  aid  the 
systems  manager  in  these  tasks.  In  the  long-range  Logistics  Management  System 
(LMS)  planning  activity  being  guided  by  Battelle,  the  above  Weapon  System 
Management  area  was  identified  as  a  first-start  area. 

It  is  clear  that  the  long-range  LMS  planning  process  will  take  at 
least  a  year  before  the  requirements  for  any  management  system  can  be  identified 
clearly  enough  to  begin  the  Data  Automation  Requirement  (DAR)  process.  Because 
of  the  initiatives  taken  by  the  Readiness  Control  Center  (RCC)  at  Ogden  in 
supporting  the  System  Management  function,  Battelle  was  asked  to  review  some 
software  Ogden  had  developed  to  determine  if  any  or  all  of  it  might  be  trans¬ 
ferable  to  other  Air  Logistics  Centers  to  enhance  the  System  Managers' 
capabilities  on  an  interim  basis. 
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In  order  to  be  considered  for  transfer,  the  software  packages  had  to 
be  fully  developed,  implemented,  documented,  and  transferable  without  signifi¬ 
cant  modification.  These  limitations  are  necessary  to  provide  the  ALC's 
System  Managers  with  near-term  capabilities  that  are  of  the  most  benefit  with 
minimum  technical  risk. 

The  procedures  developed  by  Battelle  were  designed  to  evaluate  the 
software  packages  on  the  basis  of  utility  to  the  systems  in  terms  of  both 
system  performance  and  system  management  (operational  impact)  and  the  ease  of 
ADP  transfer  (technical  risk) . 


OBJECTIVE 


The  objective  of  the  reported  effort  has  been  to  provide  a  structure 
for  the  evaluation  of  three  RCC-developed  software  packages  to  determine  if  any 
or  all  of  them  could  provide  interim  support  for  the  ALC’s  System  Managers  to 
enhance  their  capabilities  until  results  of  the  long-range  Weapon  Systems 
Management  planning  activity  (a  first  step  in  the  long-range  Logistics 
Management  System)  are  available. 


Ofte. 
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PROCEDURES 


BG  Leo  Marquez  originated  the  concept  of  providing  ALC  System 
Managers  with  interim  support,  consisting  of  software  packages  that  have  been 
developed  by  the  Ogden  ALC’s  Readiness  Control  Center  (RCC).  General  Marquez 
requested  that  Battelle  review  the  RCC  software  packages  to  determine  if  any 
or  all  of  them  were  transferable  to  System  Managers  in  the  other  ALCs.  As  a 
part  of  their  work  on  development  of  the  long-range  Logistics  Management  System 
(LMS)  planning  activity,  Battelle  responded  to  General  Marquez's  request  in 
September  1980,  visiting  the  Ogden  ALC  for  briefings  on  the  several  software 
packages  that  are  in  various  stages  of  development  there.  The  Battelle  staff 
then  reported  to  General  Marquez  on  the  candidate  packages  that  might  be 
suitable  for  transfer  to  the  other  ALCs.  Subsequently,  Battelle  was  asked 
to  pursue  the  matter  further. 

The  principal  task  was  to  establish  criteria  to  evaluate  the  software 
packages,  to  apply  these  criteria  to  the  candidate  packages,  and  to  thereby 
recommend  packages  for  transfer  to  the  other  ALCs.  It  was  decided  that  a 
review  of  the  packages  by  ALC  representatives  would  be  the  best  technique  for 
evaluating  packages  .or  transfer,  considering  these  representative's  awareness 
of  the  ALC  environment.  LOACF  and  XRB  personnel  worked  with  Battelle  staff  to 
arrange  and  schedule  the  evaluation  process.  This  schedule  is  given  in 
Appendix  A.  Each  of  the  scheduled  tasks  completed  thus  far  are  discussed  in 
the  following  sections. 


Identify  Meeting  Participants 

Each  ALC  was  asked  to  identify  an  experienced  system  manager  who 
would  be  able  to  evaluate  the  RCC  software  packages  in  terms  of  utility  to 
the  SM  and  to  provide  one  representative  for  a  meeting  scheduled  to  be  held 
at  Ogden  ALC.  The  meeting  agenda  and  participants  are  shown  in  Appendix  B. 

Define  the  Criteria  for  Transfer  Packages 

Since  the  objective  of  this  effort  was  to  provide  increased  interim 
support  for  the  SM,  the  candidate  capabilities  to  be  evaluated  were  intended 
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to  w_  fully  developed  and  documented.  The  Ogden  RCC  was  asked  to  recommend 
packages  for  evaluation,  based  on  these  overall  guidelines  and  with  an 
additional  stipulation  that  the  capabilities  be  as  universal  as  possible 
(i.e.,  not  applicable  only  to  a  select  group  of  weapon  systems). 

Identify  Existing  Hardware 


In  order  to  assess  the  initial  ADP  environment  within  which  these 
systems  would  have  to  operate  at  each  ALC,  knowledgeable  representatives  of 
the  ALCs  participated.  Their  function  was  to  evaluate  the  hardware  require¬ 
ments  associated  with  each  software  package  and  to  scope  the  degree  of  difficulty 
associated  with  the  technical  transfer.  While  hardware  was  not  to  be  the 
deciding  factor,  lack  of  its  availability  could  cause  significant  schedule 
delays  in  transferring  software. 

BCL  staff  offered  technical  assistance. and  several  of  the  ALCs 
sent  representatives  to  Ogden  ALC  to  assess  the  required  capabilities  and 
report  on  hardware  available  at  their  ALCs. 

Structure  the  Meeting 


A  two-stage  approach  was  designed  by  BCL  for  the  evaluation  process. 
The  first  stage  consisted  of  an  information  exchange  session.  Each  software 
package  was  briefly  described  by  the  Ogden  RCC  to  acquaint  the  representatives 
from  the  ALCs  with  the  overall  function  of  that  package.  The  evaluation  group 
then  split  into  two  separate  sessions.  One  group  assessed  the  package  in 
terms  of  its  operational  impact,  or  utility  to  the  System  Manager,  while  the 
other  group  assessed  the  technical,  hardware,  and  software  implications.  Each 
group  went  through  a  structured  questioning  process  that  addressed  14  infor¬ 
mation  categories,  such  as  data  structure  and  man-machine  interface,  in  some 
detail  (Appendices  C  and  D) .  Three  separate  packages  were  examined :  a  Combat 
Support  Capability  Management  System  (CSCMS) ,  a  Modification  Tracking  System, 
and  a  M1CAP  analysis  routine. 

The  second  stage  followed  the  information  exchange  sessions  on  the 
individual  packages  and  consisted  of  a  comparative  evaluation  process.  Using 
a  system  merit  evaluation  tree,  the  various  packages  were  ranked  one,  two,  and 


three;  first  in  the  area  of  technical  risk  (one  being  the  least  risk)  and  then 
in  the  area  of  performance  impact  (one  having  the  greatest  positive  impact) . 
After  ranking  the  packages  relatively,  an  absolute  evaluation  for  the  number 
one  system  was  identified. 

Following  the  meeting,  the  unweighted  calculation  of  the  performance 
and  risk  areas  was  used  to  illustrate  how  to  situate  the  packages  on  a  matrix 
of  performance/risk.  To  be  considered  for  transfer,  a  package  should  fall 
toward  the  upper- left  of  such  a  matrix.  The  results  of  this  sample  evaluation 
are  presented  in  the  following  section. 
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EVALUATION  RESULTS 


As  indicated  previously,  the  session  participants  were  requested  to 
evaluate  both  the  relative  risk  associated  with  transferring  the  three  RCC 
developed  systems  and  the  relative  performance  impact  associated  with  the 
three  systems.  The  risk  and  performance  factors  as  well  as  the  resulting 
relative  risk  and  performance  impacts  are  presented  in  Appendix  E. 

There  are  a  number  of  ways  of  using  the  information  in  Appendix  E  to 
evaluate  the  relative  merit  of  transferring  one  or  more  of  the  three  systems. 

The  actual  evaluation  is  an  AFLC  responsibility.  The  following  illustrative 
analyses  are  presented  only  to  Indicate  how  the  data  may  be  used.  In  the  first 
illustration,  no  weighting  of  absolute  risk  or  absolute  performance  impact  is 
applied.  That  is,  the  information  contained  in  the  bottom  row  of  Figures  E-2 
through  E-6  is  not  used. 

In  this  example,  the  risk  factors  are  uniformly  weighted  as 
indicated  in  Figure  1.  The  relative  risk  scores  for  each  system  for  each 
risk  factor  are  multiplied  by  the  weight  for  each  risk  factor  and  summed 
across  the  factors  to  yield  a  total  relative  risk  score. 

Similarly,  in  Figure  2,  the  performance  impact  factors  are 
uniformly  weighted  and  the  relative  performance  score  for  each  system  for  each 
performance  factor  is  multiplied  by  the  appropriate  weight.  The  uniformly  weighted 
scores  are  then  summed  across  the  performance  factors  to  yield  a  total  relative 
performance  impact  score  for  each  system. 

Relative  performance  impact  is  plotted  against  relative  risk  for  each 
system  in  Figure  3.  It  is  seen  that  the  MICAP  Analysis  System  would  have  the 
least  risk  while  the  Modification  Tracking  system  would  have  the  greatest 
performance  impact.  The  CSCMS  has  the  highest  risk  and  the  same  performance 
impact  as  the  MICAP  analysis  system. 

If  Relative  System  Merit  is  defined  as: 

Relative  System  Merit  ■  Performance  Impact  x  Risk 
the  scores  are  as  follows: 


Relative  Risk  Uniformly  Weighted  Risk 


FIGURE  1.  EVALUATION  OF  SYSTEMS'  RELATIVE  RISK 
UNDER  UNIFORM  WEIGHTING 
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FIGURE  2.  EVALUATION  OF  SYSTEMS'  RELATIVE  PERFORMANCE  IMPACT 
UNDER  UNIFORM  WEIGHTING 
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FIGURE  3.  PERFORMANCE  IMPACT  VERSUS  RELATIVE  RISK 
UNDER  UNIFORM  WEIGHTING 


11 


System 

System  Merit  Score 

CSCMS 

5.34 

Modification 

3.76 

Tracking 

MICAP  Analysis 

3.12 

Low  System  Merit  scores  are  preferred  to  high  scores  under  the 
scoring  system  used. 

A  non-uniform  weighting  scheme  that  takes  into  account  the  absolute 
risk  and  absolute  performance  impact  shown  in  the  bottom  rows  of  the  matrices 
in  Figures  B-2  through  B-6  is  analyzed  in  Figures  4  and  5.  The  non-uniform 
weighting  shows  up  in  Figure  4  in  the  Investment  Cost  Factors  and  Ease  of 
Transfer  factors,  and  in  Figure  5  in  the  SM  effectiveness  factors.  The  non-uniformly 
weighted  results  are  plotted  in  Figure  6  along  with  the  weighted  result.  The 
effect  of  the  weighting  is  to  slightly  decrease  the  relative  risk  associated 
with  all  three  systems.  The  performance  impact  of  the  MICAP  Analysis  System 
is  slightly  improved.  The  Modification  Tracking  System's  performance  remains 
unchanged  and  the  CSCMS  has  slightly  decreased  performance  impact. 

A  third  illustration  of  how  the  information  may  be  used  is  illustrated 
in  Figure  7.  In  this  example  hardware  and  facility  considerations  are  eliminated 
from  the  Relative  Risk  analysis.  This  analysis  holds  independent  of  whether 
the  absolute  risk  information  contained  in  the  bottom  rows  of  the  matrices 
in  Figure  E-2  through  E-5  are  considered.  This  result  derives  from  the  fact 
that  the  non-uniform  absolute  risk  weighting  resulted  only  from  the  hardware 
and  facility  factors  which  were  eliminated. 

The  effect,  shown  in  Figures  8  and  9  clearly  separates  the  risk 
associated  with  the  three  systems  under  uniform  or  non-uniform  weighting  while 
leaving  the  performance  impact  unchanged.  Hardware  and  facilities  do  not 
appear  among  the  Performance  Impact  factors  so  Figures  2  and  5  would  remain 
unchanged  for  this  Illustration. 
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FIGURE  4.  EVALUATION  OF  SYSTEMS’  RELATIVE  RISK 
(NON-UNIFORM  WEIGHTING) 
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FIGURE  5.  EVALUATION  OF  SYSTEMS'  RELATIVE  PERFORMANCE  IMPACT 
(NON-UNIFORM  WEIGHTING) 
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FIGURE  6.  PERFORMANCE  IMPACT  VERSUS  RELATIVE  RISK 
(NON-UNIFORM  WEIGHTING) 
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FIGURE  7.  EVALUATION  OF  SYSTEMS’  RELATIVE  RISK  WITH 
NO  HARDWARE  OR  FACILITY  CONSIDERATIONS 
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FIGURE  9.  PERFORMANCE  IMPACT  VERSUS  RELATIVE  RISK 
(NON-UNIFORM  WEIGHTING  WITH  AND  WITHOUT 
HARDWARE/FACILITY  CONSIDERATIONS) 
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The  relative  system  merit  for  the  three  examples  is  summarized 


as  follows: 


CSCMS 

Modification 

Tracking 

MICAP  Analysis 


SYSTEM  MERIT  SCORE 


All  Factors  Considered 
Uniform  Non-Uniform 

Weighting  Weighting 


Hardware  and  Facility 

Factors  Eliminated _ 

Uniform  Non-Uniform 

Weighting  Weighting 


Again,  low  Merit  Scores  are  preferred  to  high  scores. 

A  fourth  illustrative  analysis  based  on  the  ALC  session  participants' 
preferences  given  in  Table  E-7  indicates  that: 

1.  If  only  one  system  were  to  be  transferred  "as  is",  the 
Modification  Tracking  and  MICAP  Analysis  Systems  would 
be  equally  preferred  over  the  CSCMS 

2.  If  only  one  system  were  to  be  transferred  and  each 
ALC  was  allowed  to  tailor  the  system  to  its  peculiar 
requirements,  the  Modification  Tracking  System  would 
be  preferred  over  the  MICAP  and  CSCMS  systems 

3.  If  two  systems  were  to  be  transferred,  the  Modification 
Tracking  and  MICAP  Analysis  systems  would  be  chosen  with 
the  Modification  Tracking  and  CSCMS  combination  being 
preferred  as  a  close  second  choice.  These  choices 
would  hold  independent  of  whether  the  systems  were 
transferred  "as  is"  or  whether  modifications  were  permitted. 

It  must  be  remembered,  however,  that  these  choices  were  related  to 
the  cross  section  of  weapon  systems  and  areas  of  expertise  represented  by  the 
attendees.  They  should  not  be  construed  as  representing  the  official,  or 
even  necessarily  a  representative,  position  of  the  ALCs. 
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OBSERVATIONS /RECOMMENDATIONS 


The  following  observations  and  recommendations  derived  from  Battelle's 
efforts  to  facilitate  the  evaluation  which  was  designed  to  Identify  the  relative 
merits  of  transferring  one  or  more  of  the  CSCMS,  Modification  Tracking,  and 
MICAP  Analysis  Systems  from  the  Ogden  Readiness  Control  Center  to  the  other 
ALCs.  Battelle's  role  was  not  to  evaluate  the  systems  per  se,  but  rather  to 
provide  methodology  for  identifying  the  preferences  of  AFLC  personnel  from  the 
ALCs  and  for  providing  the  methodology  to  analyze  those  preferences. 

In  performing  our  role,  certain  information  was  obtained  and 
impressions  were  formed  that  are  believed  to  be  worth  documenting  as  observa¬ 
tions.  These  are  noted  below  followed  by  the  basic  recommendation. 

The  observations,  in  unranked  order,  are  as  follows: 

o  There  is  little  doubt  that  the  ALCs  could  productively  use 
the  Modification  Tracking  and  MICAP  Analysis  Systems.  They 
would,  of  course,  have  to  be  tailored  to  the  weapon  systems 
assigned  to  the  various  ALCs  to  enhance  their  usefulness  in 
their  own  work  environment. 

o  Ready  access  to  computer  facilities  is  critical  to  successful 
transfer. 

o  Analytical  capability  in  MM  at  the  ALCs  is  uneven  across 
ALCs.  Adequate  support  is  essential  to  benefit  from  the  use 
of  these  or  any  other  tools  developed  for  the  System  Manager. 

o  System  development  at  the  ALCs  is  stifled  by  lack  of  compu¬ 
tational  capability  that  is  accessible  to  analysts  for 
experimental  development . 

o  The  CSCMS  model  needs  further  development  to  handle  (a) 
multiple  systems  and  address  the  "common  item"  problem, 
and  (b)  non-tactical  weapon  systems. 

o  The  CSCMS  model  will  require  dedicated  analytical  support 
to  use  properly  (i.e.,  to  understand  underlying  assumptions 
implicit  in  the  model  and  to  generate  valid  results). 

o  The  CSCMS  model  results  need  to  be  verified/validated. 

o  The  CSCMS  model  is  a  desirable  command  capability  that 
should  be  centrally  supported  (at  HQ  AFLC?)  but  accessible 
throughout  the  command. 

o  Further  dialogue  should  be  established  among  the  SMs  at 
the  various  ALCs  so  that  cross-fertilization  of  developing 
management  system  capabilities  can  continue  to  take  place 
at  regular  intervals. 
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Finally ,  the  basic  recommendations  are: 

o  LO/XR  analyze/weigh  the  session  results  presented  in  this 
report  to  determine  the  system(s)  appropriate  for  transfer. 

o  UK  should  determine  the  most  effective  way  to  implement 
each  of  the  three  systems  at  each  of  the  ALCs  other  than 
Ogden  ALC.  This  determination  should  inlcude  consideration 
of  existing  computer  availability  as  well  as  new  purchase. 


APPENDIX  B 


MEETING  STRUCTURE  AND  PARTICIPANTS 


AGENDA  FOR  MEETING 
13-15  January  1981 
AT  OO-ALC 

DAY  1 

Welcome  >  OGDEN 
Opening  Remarks  -  AFLC/LO 
Purpose  of  the  Meeting  -  AFLC/LO 
RCC  Overview  -  OGDEN 
BREAK 

LMS  Planning  Project  -  AFLC/XR 
Evaluation  Process  -  BATTELLE 
Wartime  Capability  Assessment  Technique  -  OGDEN 
LUNCH 

PACAF  Combat  Support  and  Capability  Management 
System  Presentation  -  OGDEN 
Evaluation/Discussion  -  BATTELLE 

DAY  2 


MOD  Tracking  System  Presentation  -  OGDEN  0800-0900 

Evaluation/Discussion  -  BATTELLE  0900-1130 

LUNCH  1130-1300 

MICAP  Analysis  System  Presentation  •  OGDEN  1300-1400 

Evaluation/Discussion  •  BATTELLE  1400-1630 


0800-0810 

0810-0815 

0815-0830 

0830-0930 

0930-0940 

0940-0955 

1000-1030 

1030-1130 

1130-1300 

1300-1400 

1400-1630 


B-2 


DAY  3 


Application  of  Score  Sheets  and  Prioriti 
zation  -  BATTELLE 
Closing  Renarks  -  AFLC/LO 


0800-1145 

1145-1200 


LIST  OF  ATTENDEES 
FOR 

READINESS  CONTROL  CENTER  (RCC)  CAPABILITY  PLANNING  MEETING 


ATTENDEES 

GRADE 

ORG 

Barbara  Arnold 

GS-13 

HQ  AFLC/LO 

Fred  Healae 

LtCol 

HQ  AFLC/LO 

Duane  Tucker 

GS-13 

HQ  AFLC/XR 

Bob  Galloway 

GS-13 

SA-ALC/MMSS 

Charles  Jurek 

GS-13 

SA-ALC/MMMR 

Robin  Kagen 

GS-12 

OC-ALC/MMM 

Jim  Bias 

GS-13 

OC-ALC/MME 

howard  bright 

GS-13 

WR-ALC/MMS 

A1  HcQuary 

GS-14 

SM-ALC/MMS 

Carl  bistefano 

GS-12 

SM-ALC/MMM 

Doug  Hill 

Contractor 

Battelle 

Kay  Miller 

Contractor 

Battelle 

Mike  Kluse 

Contractor 

Battelle 

Don  Hines 

GS-11 

OO-ALC/MMS 

Gene  Jones 

GS-12 

OO-ALC/MMS 

Don  Naef 

GS-12 

OO-ALC/MMS 

Mike  Williams 

GS-12 

OO-ALC/XR 

Bob  Tripp 

Major 

OO-ALC/MMM 

Vern  Thom 

Major 

OO-ALC/MMM 

Perry  Koch 

Major 

OO-ALC/MMM 

Ken  Hales 

GS-05 

OO-ALC/MMM 

Garry  Peery 

GS-12 

OO-ALC/ACD 

Terry  Fernelius 

GS-11 

OO-ALC/ACD 

Susan  Busier 

CS-09 

OO-ALC/ACD 

Woody  Bryant 

LtCol 

OO-ALC/MMA 

RCC  SYSTEM  EVALUATION 
OPERATIONAL  IMPACT 


INFORMATION 

CATEGORY 

l - - 

CSMCS  MODEL 

(A) 

MODIFICATION  TRACE INC  SYSTEM 
(B) 

MICA?  ANALYSIS  SYSTEM 
«C) 

Bata  Structura 

Need  DIM  authorisation  levels 

Uses  Item  essentiality  codas 

Masd  LIS/SRU  relationahlpa  fron  WUC 
Source  la  ILDF  or  0029 

Source  of  LRU/SRU  nay  be  different 
based  on  life  of  weapon  system, 
l.e.,  (CIA  vs.  ms 

May  need  DG41  and  D029  depending  on 
range  of  itens  selected 

NRSK  conference  Is  starting  point 
for  lten  selection 

Need  war  plans  and  resources 
jOnly  deals  with  recoverable  sparee- 
|  about  7S0  itens  currently  selectee 
Problane  with  "squeezed"  D041  tapes 

CA26/C079  should  provide  data  but 
are  not  up  to  date 

Don't  want  to  feed  two  systems  sane 
data 

Need  SM  to  tailor  milestones  to  his 
systen/nodlficatlon 

Can  ba  used  to  highlight  need  to 
nalntain  G026/C079 

Also  uses  H0S7  data  for  financial 
tracking  systen 

1 

: 

Off  D-16SB  tapes  (monthly) 

Can  process  dally  tapes,  but  mot 
that  useful 

. 

Geographic  Distribution 
of  Data  Entry  and  Oaars 

Each  MAJCOM  nalntains  aaparate  data 
base 

Need  Information  from  1050 

AFLC  can't  access  that  data  now 

Each  M  is  different  weapon 
platform 

Most  data  located  at  sans  ALC 

May  be  competition  among  users  at 
sans  ALC — hardware  limitations 

Soma  problem  arc  actually  commend 
bound,  not  supply 

! 

Criticality  of  Data 

Could  replace  Ml,  2,  3  aysten 
Collected  every  30  days 

Must  be  credible  to  use  with  XM  or 
no  Inpact 

Provide  aaar-tlna  but  no  real  time 
visibility 

"As  of"  date  la  used  for  data,  not 
report  generation 

Does  mot  track  field  performance  to 
check  actual  progress 

Talk  to  engineers  co  gat  data 
manually 

Accuracy  critical  for  FMS  use 

Archival  Conaldaratlona 

Currently  use  nost  recent  quarter 
only 

Need  trend  data  to  predict  problems 
on  cop  20-25  NMCS 

Need  to  store  only  auanary  dsta, 
not  raw  input 

Car.  use  history  by  tall  nuebar  for 
configuration  control 

No  trend  data  currently  available 
Repast  milestone  events  should  bo 
Totainod— problca  areaa 

Trend  analysis  is  port  of  current 
business 

Need  history  of  at  least  tbs  top  20 

Processing  Requirements 

Must  understand  assumptions  such  as 
1002  cannibal itat ion 

Assumes  consolidation  of  shortages 
Each  base  has  sane  NOES  rata 
Evaluates  depth  of  WEEK,  not  range 

Modification  lnterrelatlonshipe  ore 
not  shown  (a.g.,  do  this  TCTO 
first) 

Dose  standard  events  and  tints,  but 
can  be  edited 

Shows  olipo 

Flexible  calendar 

Longesc  tine  roquirenent  la  Co 
Identify  top  20 

Lots  of  conputetlonal  time,  but 
not  much  coop lenity 

Input /Output  Volume 

Froblam  in  F-16  Is  not  knowing  what 
question  to  ask 

Esquire  full  understanding  of 
eystam  capability  to  analyte 
output 

Need  support  of  upper  etaff  wanting 
the  model 

Esquired  skilled  aanpower 

Meed  person  dedicated  to  input  data 
—all  engineers  can't  operate  and 
understand 

Will  glva  summit  Ion  to  require 
C026/79  update 

Consumes  much  time  for  Initial 
load/nodlflcation 

i 

Could  save  manpower  In  preparation 
of  charts  and  In  data  search 

Takes  3  hours  to  load  data,  * 
hours  to  run  monthly  require¬ 
ments 

Output  Requirements/ 
Response  Tinea 

i 

lest  used  for  "what  if"  requests— 
will  probsbly  Increase  if 
capability  available 

Frobably  generate  every  30  days  on 
regular  basis 

Ihould  Improve  quality  of  "what  if" 

fMfOMfi 

Utility  la  dealing  with  M  depends 
an  credibility  of  nodal 

LOC  officer  probably  closer  to  MM) 
schedule  than  aodlflcstlon 
manager 

System  will  benefit  F-15/C-130 
Review  selected  modifications  with 
branch  chief  2X/aonth 

Review  oil  about  once/quortor 

High  ust  aysten 

Rspresente  Information  on  a  major 
rating  sres 

Sencratsa  13  different  report 
typos 

RCC  SYSTEM  EVALUATION 
OPERATIONAL  IMPACT 


nrontATiM 

CATKOKY 

C3MCS  MODEL 
<A> 

MODIFICATION  TIACKIHC  SYSTEM 
(D) 

MICAP  ANALYSIS  SYSTEM 
(C) 

Needed  Interface  with 
Other  Systems  and 
Volume  of  Date  to  be 
Passed 

002V  baa  all  elenenta  you  need 

DIM,  DM1  nay  be  better  aourcea 
Deed  1050  Interface 

Doean't  allow  alauitaneoua  analyala 
of  acenarloa  or  weapon  ayatena  to 
give  accurate  calculation  of 
cone  no  ltuo  usage 

Mead  autonatic  extraction  of  data 
[  from  other  ayateae 

C02A/79  should  be  updated  dally 

Don't  need  Interface  this 
frequently 

Should  got  total  tape  and  extract 
data 

CO 79  Is  only  adequate  at  acnlanoual 
review  tlM 

Interfaces  with  DliSM 

Some  would  prefer  Interface  with 
dally  tepee 

’ 

Man-Machine  Interface 

Meade  aaalyat  to  interpret 
quest Iona  and  tranafora  into 
prograai  input 

Meeda  analyat  to  do  pralinlnary 
analyala  of  output 

Mot  available  directly  to  SM 

Need  dedicated  people  for  Input 

Meed  to  Input  and  aalntaln  la 
different  than  need  to  extract 

data 

Meeds  to  be  usable  by  SM— not 
trained  analysts 

Can  be  used  to  detect  errors  In 

D-165  Input 

Training  gequlrcnenta 

Can't  afford  training  tine 

Meed  good  SM  annual 

Naed  ayaten  In  SM  ares-ainplif led 
Instructions 

gepatate  manuals  for  users  and 
operators 

SM  doesn't  need  to  know  how  to 
input,  but  slaply  how  to  extract 

Availability  of  Syeten 

Mead  ayatan  In  SM  area,  not  In 
another  building 

"What  ifa"  happen  anytine — have  to 
bo  able  to  respond 

Meed  ayaten  in  SM  area,  not 
centrally  located 

Heed  syeten  In  SM  mren 

Can  do  manually  If  ayatan  la  down, 
but  nore  tine  consuming,  less 
accurate 

Vulnerability 

Conalda rations 

Want  only  ay  syeten— no  Input  fron 
other  systene 

Must  have  standardization  of  core— 
with  flexibility  to  adapt  to 
specific  SM  needs 
for  local  control,  uae  password 

SM  has  to  determine  appropriate 
milestones  for  his  nodlficatlon 

No  probleos  seen 

Security /Privacy 
Conalderationa 

No  classified  data 

got  applicable 

APPENDIX  D 

TECHNICAL  VIEWPOINT  ON  SYSTEMS  EVALUATED 


RCC  SYSTEM  EVALUATION 
TECHNICAL  VIEWPOINT 


INFORMATION 

CATEGORY 

C3MCS  MODEL 
(A) 

MODIFICATION  TRACKINC  SYSTEM 
<B) 

MICAP  ANALYSIS  SYSTEM 
(C) 

Data  Structure 

Data  la  available  for  tactical 
eyatena,  but  may  not  exlet  or  nay 
be  difficult  to  obtain  for  a 
MAC/SAC  ayatem 

Data  require!  12M  bytaa  of  on-line 
dlac  apace 

Tape  Inputa 

-  D029 

-  Minimum  backorder  analyala 
technlquea  ayatem  data  file 

-  Operational  tracking  and 
control  eubayeten  data  fila 

Card  Input 

-  Combined  baae  and  CIRF  atock 

-  Baae  atock 

-  CIRF  atock 

Uaer  reaponaea 

Requlrea  AFLC  Forma  192D  and  A8  for 
for  each  modification 

All  data  la  Input  manually  via 
terminal 

Data  la  available  at  all  ALCa 
Built-In  audit  trail  capability 

Data  baae  protected  agalnat 
achedule  changes 

Data  baae  la  a  PDP-11/70  Index 
acquentlal  dlac  file 

Data  consiatq  of  D16SB  tape  and 
user  reaponaea  to  program  prompta 
Data  available  at  all  ALCa 

Data  baae  la  composed  of  PDP-11/70 
sequential  and  Index  sequential 
dlac  files 

Data  baae  requlrea  17H  bytes  of 
on-line  disc  space 

Geographic  Dlatrlbutlon 
of  Data  Entry  and  Uaera 

Each  ALC  would  prefer  own  model 
rather  than  tie-in  to  Ogden  DEC  10 
Ogden  la  people-limited  to  aupport 
other  ALCa  on  lta  DEC  10 

Currently  no  comninicatlon  link! 

Into  Ogden  DEC  10  alnce  It  la 
located  In  a  aecura  area 

ALCa  prefer  HQ/XXS  maintain  config¬ 
uration  management  rather  than 

AF  Dealgn  Center 

Data  entry  at  the  ALC  managing  the 
modification 

Poaalble  requirement  for  data  at  HQ 
If  they  want  reports 

Very  little  resource  data — ALCa 
would  like  resource  data  Included 

Uaera/data  all  local  to  each  ALC 

Need  exlats  for  terminals  at  each 

ALC 

Criticality  of  Data 

Data  la  alwaya  not  current  nor 
perfect 

D041  data  la  alwaya  changing 
Aaaumptiona  in  model  arc  very 
Important  and  mu at  be  understood 

System  allows  one  day  to  fix  errors 
—after  one  day,  entries  arc 
regarded  as  actlona  and  are 
tracked  by  the  system 

System  provides  some  logical/ 
consistency  checks 

Local  permanent  file  maintenance 

Depends  on  other  systems  being 
streamlined 

Other  system  status  may  preclude 
usage 

Syatem  may  fade  away  during  war 

rt^ulrcd 


Archival  Consldarations 


pgden  parforaa  routine  permanent 
flic  maintenance  to  back  up  dlac 
fllaa 

{Local  permanent  (lie  maintenance 
would  be  required  at  each  ALC 


Ogden  parforaa  routine  permanent 
file  maintenance  to  back  up  disc 
filer 

[Local  permanent  file  maintenance 
would  be  required  at  each  ALC 


[Ogden  perform  routine  permanent 
file  maintenance  to  back  up  dlac 
file! 

Local  permanent  file  maintenance 
would  be  required  at  each  ALC 


Processing  Acquirement! 


(Preprocessor  on  PDA-11/70 

Model  runa  on  DEC  10 
preprocessor  prograaa  are  top-down 
etruetured  ANSII  ataadard  COBOL 

prograaa 

pr  naaetrlca  model  written  In 
top-down  IATFOB 
band  runa  model  on  VAX  11/780 
pany  problems /inconi It tenc lei  with 

data  aourcea 

p’reprocenor  require!  64K  bytea  of 
waory 

|Preproceaaor  conpoaed  of  29  COBOL 

prograaa 

|No  apeclallied  data  baaa  management 
ayataa 


Rune  on  PDP-11/70 
Approximately  20  program!  In  the 
ayataa.  One  written  in  COBOL, 
remainder  In  FORTRAN  IV  PLUS 
Prograaa  are  modular  but  not 
atructured 
{Much  character  manipulation 
Plot  10  graphic!  package  required 
to  drive  TEKTRONIX  termlnala 
[Memory  requirement!  currently 
unknown 

{Mo  apodal  laed  data  baaa  management 
ayataa 


Runa  on  PDP-11 /70 

Syetem  conelata  of  19  program: 

-  4  FORTRAN 

-  1  COBOL 

-  U  BASIC  plua  2 

BASIC  program!  being  rewritten  In 
COBOL 

Require!  6AK  bytea  of  memory 
Plot  10  graphlca  package  required 
to  drive  TEKTRONIX  termlnala 
No  apeclallied  data  baae  manage¬ 
ment  ayatem 


Input /Output  Volume 


(All  Inputa  on  "a!  required"  baala 
preproceaaor  requlraa  A  runa  on 
*aa  required"  beale 
|No  operation!  eupport— uaer  hanga 
own  tapea  and  mount!  own  packa 


baala 


tarda  and  tape  input 


Ull  Input  via  terminal 
[Weakly  update  of  data  baaa 
Initial  data  load  requlrea  approxi¬ 
mately  8  houra  per  modification 
ppdate  requlrea  approximately 

1  bear  per  week  to  update  aodlfl- 
eetleaa 


D18SB— one  reel  per  weapon  ayatem 

Ho  apedal  proceaalng 

13  runa  on  "ae  required"  baala 

1  monthly  run 

1  daily  run 

38  output  eptiooa 


RCC  SYSTEM  EVALUATION 
TECHNICAL  VIEWPOINT 


INFORMATION 

CATKCORT 

CSMCS  MOOSL 

(A) 

MODIFICATION  TRACKING  SYSTEM 
(»> 

KICAP  ANALTglS  ST STEM 
(C) 

Output  Requirement*/ 
Reeponae  Tine* 

No  special  processing  required 
On-line  and  line  printer  output 
available 

No  graphics  output — standard  writes 
to  the  terminal 

No  special  hardware  required 

Output  la  preprocessor  natrlx 

In-line  and  line  printer  output 
available 

TEKTRONIX  terminal  required  for 
graphics 

SM  should  see  data  weekly — modifi¬ 
cation  nanager  more  often 

Color  output  requires  TEKTRONIX  AO 21 
color  graphics  terminal 

All  output  via  Interactive 
terminal 

TEKTRONIX  terminal  required  for 
graphics 

38  output  options 

Ittdid  Interface  to 
Other  Systems  and 
Volume  of  Data  to  be 
Passed 

Data  frun  D029,  alnlnun  backorder 
analysis  techniques  systen  and 
operational  tracking  and  control 
systen  contained  on  tape 

Saae  and  CIRF  stock  contained  on 
cards 

No  automated  Interfaces 

Currently  no  interfaces 

Plans  to  interface  wlch  H057,  G026, 
and  G079 

H057,  C026,  and  G079  need  improve¬ 
ment  a  to  make  them  more  current 
before  Interface  Is  performed 

Only  Interface  la  with  D16SB  tape 
for  the  weapon  system 

Kan-Machine  Interface 

Operations  annual  and  maintenance 
annual  available  for  preprocessor 
Only  trivial  dociaaentation  currently 
available  for  Dynaaetrics  aodel 
Interactive  portions  are  straight¬ 
forward 

COBOL  dictates  AC  Involvement  at 

ALCs  If  system  exported 

No  special  hardware  requirements 

User  Interfaces  with  tapes,  cards, 
diac  packs,  and  interactive 
terminals 

Color  output  requires  TEKTRONIX  A021 
color  graphics  terminal 

Contractor  documentation  consists 
of  user  and  prograamcr's  manuals 
plus  data  layouts;  documentation 
la  very  hard  to  follow  and  poorly 
written — Inadequate  at  best 

User  Interfaces  with  AFLC  Forms 

192D  and  A8  and  with  interactive 
terminal 

COBOL  dictates  AC  involvement  at 

ALCs  If  system  exported 

User  Interfaces  with  D165B  tape 
and  Interactive  terminal 

Operators  manual  with  minimal 
flowcharts  available 

Interactive  portions  are  straight¬ 
forward 

COBOL  dictates  AC  Involvement  at 

ALCs  If  system  exported 

Training  laqulrement* 

Full  understanding  of  Dyne metrics 
aodel  would  require  several 
months  of  concentrated  effort 
Model  could  be  used  after  two  weeks 
Of  study  but  with  very  limited 
understanding 

Inquires  involvement  of  operations 
research  analyst  in  addition  to 
a  skilled  programmer 

Contractor  support  available 

Ogden  currently  supporting  system 
with  two  programmers  and  having 
difficulty 

No  contractor  support 
bgden  has  coaailtted  6  to  10  nan- 
weeks  of  effort  to  date,  and 
estimates  an  additional  2  to  3 
man-months  required  before  full 
utility  of  system  la  realized 

Mlnlnlnal  training  required  for 
usage 

One  day  training  required  for  full 
operation 

Terminal  interrogations  are  very 
easy  to  understand 

Availability  of  Syataa 

ALCs  other  thllh  Ogden  save  no  excesi 
computing  capacity  available  to 
to  host  the  system 

Currently  no  capability  to  process 
classified  data 

System  downtime  would  not  cause  a 
major  disruption  In  functioning 
of  this  work  area 

ALCs  other  than  Ogden  have  no 
excess  computing  capacity  avail¬ 
able  to  host  the  system 

System  downtime  not  critical  for 
aircraft  but  more  Important  for 
missiles  due  to  higher  priority 
Terminals  not  available  at  ALCs 
other  than  Ogden 

ALCs  other  than  Ogden  have  no 
excess  computing  capacity  avail¬ 
able  to  hose  the  system 

Systen  downtime  not  critical 
Terminals  not  available  at  ALCs 
other  than  Ogden 

Vulnerability 

Considerations 

Fermaoent  file  maintenance  and 
backup  procedures  required  at  all 
ALCs  hosting  system 

Permanent  file  maintenance  and 
backup  procedures  required  at  all 
ALCs  hotting  system 
ianual  procedures  always  available 
for  backup 

Permanent  file  maintenance  and 
backup  procedures  required  at 
all  ALCs  hosting  system 

D16SB  tapes  always  available 

Security /Privacy 
Considerations 

Scenarios  may  be  classified- 
dictates  encoding  of  date  on  a 
secure  faclllty/computer; 
classified  scenarios  prohibit 
other  ALCs  from  tie-in  to  Ogden 

Not  applicable-all  data  unclassi¬ 
fied  on  modifications 

Not  applicable — all  data 
unclassified 

Spates  Development 

Make  adaptations  necessary  to 
consider  strategic  as  well  as 
tactical  syataas 

Desirable  to  conelder  conmunlcotloni 
and  electronics  as  well  as  weapon 

systems 

Following  future  capabilities 
suggested: 

-  Exception  reporting 

■  -  Tail  number  tracking 

-  letter  prompts 

-  Closed  loop  with  standard 

systems 

-  Batter  user's  manual 

-  Applicability  to  coammlca 

tlone  and  electronics 
modification  management 

BASIC  plus  2  routines  being 
rewritten  In  COBOL 

Local  (Ogden)  enhancements 
planned— files  being  reorganised 
to  make  run  time  faster 

APPENDIX  E 


EVALUATION  OF  SYSTEM  MERIT 


EVALUATION  OF  SYSTEM  MERIT 


The  relative  merit  of  the  three  systems: 

o  Combat  Support  Capability  Management  System  (CSCMS) 
o  Modification  Tracking  System,  and 
o  MICAP  Analysis  System 

under  consideration  for  transfer  to  ALC's,  was  evaluated  along  two  major 
dimensions.  These  were: 

Relative  Risk,  and 
Performance  Impact. 

These  major  dimensions  were  further  broken  down  into  the  component  factors 
shown  in  Figures  E-l  through  E-6.  Figures  E-2  through  E-5  were  used  by  the 
session  participants  to  score  the  relative  risk  associated  with  transferring 
each  of  the  three  systems  under  consideration. 

The  set  of  risk  Impact  factors  were  selected  to  provide  a  reasonably 
comprehensive  cross-section  of  the  elements  that  might  affect  the  ability 
of  AFLC  to  successfully  transfer  the  candidate  systems  from  the  Ogden  Readiness 
Control  Center  to  the  other  ALC’s. 

A  three-point  scoring  system  was  used  to  evaluate  relative  risk  with 
a  score  of  1  indicating  the  least  risk  and  a  score  of  3  the  greatest  risk. 

After  filling  in  the  relative  risk  in  each  matrix  the  participants 
were  asked  to  evaluate  the  absolute  risk  (high,  medium,  or  low)  associated 
with  the  least  risky  system  in  each  risk  category.  These  evaluations  are 
indicated  in  the  bottom  row  of  each  matrix. 

The  performance  factors  shown  in  Figure  E-6  were  designed  to 
facilitate  the  session  participants  judging  the  relative  utility  of  the 
candidate  systems  insofar  as  their  transfer  would  impact  either  weapon  system 
performance  or  the  system  manager's  performance. 

Figure  E-6  indicates  the  relative  performance  impact  factors  and 
scores  associated  with  each  system.  Again  a  three-point  scoring  system  was  used. 
In  this  case  a  score  of  1  indicates  a  higher  impact  on  improved  performance 
than  a  2  or  a  3. 


RELATIVE  RISK 


TECHNICAL  RISK: 


FIGURE  E-2.  PRIMARY  RISK  FACTORS 


FIGURE  E-3,  PRIMARY  RISK  FACTORS  (Continued) 


SCHEDULE  RISK: 
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FIGURE  E-A.  PRIMARY  RISK  FACTORS  (Continued) 


♦From  Configuration  Control  Viewpoint 


E-8 


Analogously  to  the  case  for  the  risk  assessment,  the  participants 
were  asked  to  rate  the  absolute  performance  improvement  (high,  medium,  or  low) 
associated  with  the  system  having  the  most  performance  Impact  (lowest  relative 
score)  in  each  performance  category.  These  evaluations  are  given  in  the  bottom 
row  of  the  matrix  in  Figure  E-6 . 

After  filling  in  the  matrices  to  evaluate  relative  risk  impact  and 
relative  performance  impact  for  each  system,  the  ALC  representatives  at  the 
session  were  asked  to  identify  which  system  they  would  choose  if  they  could 
have  only  one  of  the  systems.  This  question  was  to  be  answered  independent  of 
hardware  requirements  and  first  assuming  that  the  system  was  transferred  as 
it  exists  today  at  the  RCC.  They  were  then  asked  to  choose  one  system  but 
assume  that  only  the  generic  capability  would  be  transferred  and  that  the 
ALC  would  be  allowed  to  tailor  the  system  to  its  own  system  peculiar  requirements. 

The  ALC  representatives  were  then  asked  to  choose  the  two  systems 
that  they  would  most  like  to  have  transferred.  Again  they  were  to  choose 
two  systems  assuming  the  systems  would  be  transferred  "as  is”  and  then  to 
choose  under  the  assumption  that  the  generic  capabilities  would  be  transferred 
but  that  the  ALC's  would  be  allowed  to  tailor  the  systems  to  their  ALC 
peculiar  requirements.  Again,  the  choices  were  to  be  made  independent  of 
hardware  considerations. 

The  resulting  choices  are  shown  in  Figure  E-7. 
were  to  be  non-attributive,  ALC's  are  not  identified. 


Since  the  choices 
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